Apache crashes relatively often

--001636c5ad4dc85b16046c8f67f4
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hey guys,

We are having weird memory leak issues with Apache and some mod_perl.
Basically what happens is that, depending on load, we eventually run out of
ram and apache crashes. We have the setting that will restart apache after
a given numer of requests, and that seems to work temporarily, but
eventually in the log it claims to restart, but no ram is freed. The
following is written to the log when that happens:


[Wed Jun 17 13:07:49 2009] [notice] Child 5056: Process exiting because it
> reached MaxRequestsPerChild. Signaling the parent to restart a new child
> process.
> [Wed Jun 17 13:07:49 2009] [notice] Parent: Received restart signal --
> Restarting the server.
> [Wed Jun 17 13:07:49 2009] [warn] (OS 10038)An operation was attempted on
> something that is not a socket. : setsockopt(SO_UPDATE_ACCEPT_CONTEXT)
> failed.
> [Wed Jun 17 13:07:50 2009] [notice] Child 5056: Released the start mutex
> [Wed Jun 17 13:07:50 2009] [notice] Apache/2.2.11 (Win32) mod_ssl/2.2.11
> OpenSSL/0.9.8i mod_perl/2.0.4-dev Perl/v5.10.0 configured -- resuming normal
> operations
> [Wed Jun 17 13:07:50 2009] [notice] Server built: Dec 10 2008 00:10:06
> [Wed Jun 17 13:07:50 2009] [notice] Parent: Created child process 3164
> [Wed Jun 17 13:07:51 2009] [notice] Child 3164: Child process is running
> [Wed Jun 17 13:07:51 2009] [notice] Child 3164: Acquired the start mutex.
> [Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting 64 worker threads.
> [Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting thread to listen
> on port 443.
> [Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting thread to listen
> on port 80.
> [Wed Jun 17 13:08:00 2009] [notice] Child 5056: All worker threads have
> exited.
> [Wed Jun 17 13:08:00 2009] [notice] Child 5056: Child process is exiting
>

And just to be clear, here are the relevant conf sections:

ThreadLimit 64
> ThreadsPerChild 64
> MaxRequestsPerChild 10000
> MaxMemFree 51200
>

That's in the "root" part of the config, or to be more explicit, it's not in
a virtual host or anything like that.

We are running apache on a Windows server, so if that could be part of the
issue let me know.

Any help at all would be very much appreciated.

--
fREW Schmidt
http://blog.afoolishmanifesto.com

--001636c5ad4dc85b16046c8f67f4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hey guys,

We are having weird memory leak issues with Apache and som=
e mod_perl.=A0 Basically what happens is that, depending on load, we eventu=
ally run out of ram and apache crashes.=A0 We have the setting that will re=
start apache after a given numer of requests, and that seems to work tempor=
arily, but eventually in the log it claims to restart, but no ram is freed.=
=A0 The following is written to the log when that happens:

gin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;" class=3D"gmail_quote">[Wed Jun =
17 13:07:49 2009] [notice] Child 5056: Process exiting because it reached M=
axRequestsPerChild. Signaling the parent to restart a new child process. >

[Wed Jun 17 13:07:49 2009] [notice] Parent: Received restart signal -- Rest=
arting the server.
[Wed Jun 17 13:07:49 2009] [warn] (OS 10038)An operat=
ion was attempted on something that is not a socket.=A0 : setsockopt(SO_UPD=
ATE_ACCEPT_CONTEXT) failed.


[Wed Jun 17 13:07:50 2009] [notice] Child 5056: Released the start mutex >[Wed Jun 17 13:07:50 2009] [notice] Apache/2.2.11 (Win32) mod_ssl/2.2.11 O=
penSSL/0.9.8i mod_perl/2.0.4-dev Perl/v5.10.0 configured -- resuming normal=
operations


[Wed Jun 17 13:07:50 2009] [notice] Server built: Dec 10 2008 00:10:06
[=
Wed Jun 17 13:07:50 2009] [notice] Parent: Created child process 3164
[W=
ed Jun 17 13:07:51 2009] [notice] Child 3164: Child process is running


[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Acquired the start mutex. r>[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting 64 worker thread=
s.
[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting thread to li=
sten on port 443.


[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting thread to listen o=
n port 80.
[Wed Jun 17 13:08:00 2009] [notice] Child 5056: All worker th=
reads have exited.
[Wed Jun 17 13:08:00 2009] [notice] Child 5056: Child=
process is exiting

gin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;" class=3D"gmail_quote">ThreadLim=
it 64


ThreadsPerChild 64
MaxRequestsPerChild 10000
MaxMemFree 51200
ockquote>
That's in the "root" part of the config, or to b=
e more explicit, it's not in a virtual host or anything like that.



We are running apache on a Windows server, so if that could be part of =
the issue let me know.

Any help at all would be very much appreciate=
d.

--
fREW Schmidt
ishmanifesto.com" target=3D"_blank">http://blog.afoolishmanifesto.com r>



--001636c5ad4dc85b16046c8f67f4--

RE: Apache crashes relatively often

--_0b36ef66-9c57-4359-8bba-884400c9648b_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable





My resume copies are at=20

http://www.gecpvt.com/IpResume55.pdf
http://www.gecpvt.com/IpResume55.docx

Phone numbers are

408-329-7307=2C212-203-0684=2C512-524-3905. (CELL) 512-773-8922

srinivas






From: frioux@gmail.com
Date: Wed=2C 17 Jun 2009 13:24:24 -0500
Subject: Apache crashes relatively often
To: modperl@perl.apache.org

Hey guys=2C

We are having weird memory leak issues with Apache and some mod_perl. Basi=
cally what happens is that=2C depending on load=2C we eventually run out of=
ram and apache crashes. We have the setting that will restart apache afte=
r a given numer of requests=2C and that seems to work temporarily=2C but ev=
entually in the log it claims to restart=2C but no ram is freed. The follo=
wing is written to the log when that happens:





[Wed Jun 17 13:07:49 2009] [notice] Child 5056: Process exiting because it =
reached MaxRequestsPerChild. Signaling the parent to restart a new child pr=
ocess.


[Wed Jun 17 13:07:49 2009] [notice] Parent: Received restart signal -- Rest=
arting the server.
[Wed Jun 17 13:07:49 2009] [warn] (OS 10038)An operation was attempted on s=
omething that is not a socket. : setsockopt(SO_UPDATE_ACCEPT_CONTEXT) fail=
ed.


[Wed Jun 17 13:07:50 2009] [notice] Child 5056: Released the start mutex
[Wed Jun 17 13:07:50 2009] [notice] Apache/2.2.11 (Win32) mod_ssl/2.2.11 Op=
enSSL/0.9.8i mod_perl/2.0.4-dev Perl/v5.10.0 configured -- resuming normal =
operations


[Wed Jun 17 13:07:50 2009] [notice] Server built: Dec 10 2008 00:10:06
[Wed Jun 17 13:07:50 2009] [notice] Parent: Created child process 3164
[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Child process is running


[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Acquired the start mutex.
[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting 64 worker threads.
[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting thread to listen o=
n port 443.


[Wed Jun 17 13:07:51 2009] [notice] Child 3164: Starting thread to listen o=
n port 80.
[Wed Jun 17 13:08:00 2009] [notice] Child 5056: All worker threads have exi=
ted.
[Wed Jun 17 13:08:00 2009] [notice] Child 5056: Child process is exiting



And just to be clear=2C here are the relevant conf sections:

ThreadLimit 64


ThreadsPerChild 64
MaxRequestsPerChild 10000
MaxMemFree 51200

That's in the "root" part of the config=2C or to be more explicit=2C it's n=
ot in a virtual host or anything like that.



We are running apache on a Windows server=2C so if that could be part of th=
e issue let me know.

Any help at all would be very much appreciated.
--=20
fREW Schmidt
http://blog.afoolishmanifesto.com

____________________________________________________________ _____
Hotmail=AE has ever-growing storage! Don=92t worry about storage limits.
http://windowslive.com/Tutorial/Hotmail/Storage?ocid=3DTXT_T AGLM_WL_HM_Tuto=
rial_Storage_062009=

--_0b36ef66-9c57-4359-8bba-884400c9648b_
Content-Type: text/html; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable









My resume copies are at

http://www.gecpvt.com/IpResume5=
5.pdf
http://www.gecpvt.com/IpResume55.docx

Phone numbers are
=

408-329-7307=2C212-203-0684=2C512-524-3905. (CELL) 512-773-8922

=
srinivas

Re: Apache crashes relatively often

On Wed, Jun 17, 2009 at 2:24 PM, fREW Schmidt wrote:
> We are having weird memory leak issues with Apache and some mod_perl.
> Basically what happens is that, depending on load, we eventually run out =
of
> ram and apache crashes.=A0 We have the setting that will restart apache a=
fter
> a given numer of requests, and that seems to work temporarily, but
> eventually in the log it claims to restart, but no ram is freed.

I haven't seen this before, but I'm guessing that it has to do with
the Windows MPM and the nature of Perl threads. Is it possible for
you to run your production server on Linux with the prefork MPM
instead?

- Perrin

Re: Apache crashes relatively often

Perrin Harkins wrote:
> On Wed, Jun 17, 2009 at 2:24 PM, fREW Schmidt wrote:
>> We are having weird memory leak issues with Apache and some mod_perl.

When you mention "some mod_perl", can you be a bit more specific about
which perl modules you are using ?
I have had problems of the same nature with some modules, particularly
some that have to do with XML processing; not only under Apache, but
also when used in other programs running as "daemons" for extended
periods of time. In some cases, I ended up extracting those pieces and
running them as external perl programs, via do() or system(). Not very
nice, but it stopped the problem in its tracks.

A CPAN module which helps track down such issues is Devel::Leak.